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FOREWORD 


This  report  summarizes  development  of  new  Army  aptitude  area  (AA)  composites  at  the 
Selection  and  Assignment  Research  Unit  of  the  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  over  the  1995  -  2000  period.  This  work  is  one  strand  of  a  larger  effort  to  improve 
classification  (i.e.,  determination  of  most  appropriate  initial  training  and  subsequent  job  assignment)  of 
new  recruits  entering  the  Army,  and  has  been  conducted  with  the  sponsorship  of  the  Enlisted 
Accessions  Division  of  the  Directorate  of  Military  Personnel  Management  of  the  Office  of  the  Deputy 
Chief  of  Staff  for  Personnel. 

The  AA  composites  are  derived  from  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB)  given  to  all  applicants,  and  play  a  central  role  in  defining  and  measuring  recruit  aptitudes 
required  for  training  and  assignment  into  Army  jobs.  The  report  first  describes  development  of  17  new 
composites  and  job  families,  designed  to  be  more  classification-efficient  than  the  existing  nine  AA 
composites.  The  new  17  composites  are  being  evaluated  and  tested  at  the  present  time,  and  are  under 
consideration  for  implementation  in  the  2004  -  2005  period.  The  report  then  describes  the 
development  of  a  set  of  nine  interim  composites  scheduled  to  replace  the  existing  nine  composites  in 
January  2002.  The  immediate  catalyst  for  the  development  and  implementation  of  the  interim 
composites  has  been  the  DOD  decision  to  eliminate  two  subtests  within  the  ASVAB,  thereby  creating 
a  need  to  reconfigure  the  composites.  Like  the  proposed  new  17  composites,  the  interim  composites 
are  based  on  measures  of  soldier  job  performance  and  use  all  the  information  contained  in  the  ASVAB. 

The  interim  composites  were  in  fact  implemented  on  January  2,  2002.  Prior  to  the 
implementation  decision,  briefings  and  discussions  were  held  with  the  responsible  Army  managers 
within  Total  Army  Personnel  Command,  Office  of  the  Deputy  Chief  of  Staff  for  Personnel,  and  the 
Army  Development  Systems  XXI  Task  Force. 
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Executive  Summary 


The  Army  currently  employs  nine  Aptitude  Area  (AA)  composites  to  classify  new  recruits;  they  are 
derived  from  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  subtests  in  a  manner  which 
makes  them  easy  to  calculate  but  inefficient  for  classification.  The  Office  of  the  Secretary  of  Defense 
(OSD)  is  eliminating  the  two  timed  subtests  in  the  ASVAB  by  December  31,  2001.  This  is  due  to  the 
expenses  involved  in  administering  these  tests  in  an  automated  environment.  When  this  comes  to  pass, 
the  classification  efficiency  of  the  existing  Army  composites  will  be  further  reduced  and  will 
necessitate  measures  to  redefine  the  existing  composites.  In  their  place  the  Army  is  expected  to  adopt 
new  composites,  which  have  been  developed  by  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  (ARI)  based  on  a  job  performance  criterion. 

The  Army’s  current  set  of  nine  composites  were  formed  in  1976  to  represent  measures  of  the  aptitudes 
/  skills  required  for  training  and  assignment  to  the  corresponding  nine  Army  job  families. 

Classification  efficiency  requires  the  use  of  composites  that  can  distinguish  how  well  a  recruit  is  likely 
to  perform  in  different  jobs.  The  AA  composites  have  limited  ability  to  do  this.  In  addition,  the  AA 
composites  were  not  developed  with  an  on-the-job  performance  criterion.  The  training  criteria  used  to 
validate  the  AA  composites  were  the  best  available  criteria  at  the  time,  but  do  not  fully  represent 
soldier  job  performance.  Each  composite  is  formulated  as  a  combination  of  four  unit-weighted 
subtests  (except  for  Clerical).  This  approach  is  a  carry-over  from  the  1950s  when  calculations  were 
kept  as  simple  as  possible,  reflecting  earlier  limitations  of  computing  power.  One  consequence  is  that 
AA  composites  do  not  track  performance  as  well  as  they  might.  A  better  approach  is  full  least-squares 
estimation,  in  which  all  subtests  contribute  to  each  composite  in  proportion  to  their  power  to  explain 
variation  in  performance. 

Based  upon  extensive  research  utilizing  job  performance  data,  ARI  has  developed  a  set  of  17 
operational  classification-efficient  job  families  and  corresponding  composites  which  would  be  used  for 
administrative,  counseling,  and  school  proponent  purposes.  Equivalent  minimum  eligibility  standards 
(cutoff  scores)  would  be  established  against  these  new  composites.  This  job  family  structure  is 
consistent  with  the  career  management  fields  currently  used  by  the  Army  and  with  the  current  AA 
system.  The  new  structure  is,  in  effect,  a  further  shredding  of  existing  families. 

In  early  2001  the  responsible  Army  agencies  indicated  that  the  changes  in  databases  and  procedures  to 
effect  a  change  to  17  job  families  and  composites  could  not  be  accomplished  in  time  to  meet  the  OSD 
deadline.  Accordingly,  ARI  was  tasked  to  develop  an  interim  set  of  composites  that  retained  the 
existing  nine  operational  job  families  and  utilized  the  reduced  ASVAB.  Given  the  strong  preference 
for  predicted  performance  composites  by  ARI  and  its  DCSPER  sponsor,  ARI  estimated  least-squares 
composites  using  the  1989  Skill  Qualifications  Test  (SQT)  database. 

The  resulting  interim  composites  represent  a  predicted  performance  metric  with  strong  relationships 
between  the  composites  and  AFQT;  its  application  leads  to  compositional  effects  that  favor 
qualification  rates  of  Category  1-fflA’s  relative  to  IIIB’s  and  IV’s.  As  a  new  metric,  the  interim 
composites  require  (small)  changes  in  cutoff  score  levels  in  order  to  maintain  the  existing  demographic 
balance  in  qualification  rates.  Because  the  average  predicted  performance  of  those  qualifying  under 
the  interim  composites  is  somewhat  higher  than  under  the  existing  composites,  cutoff  score  levels  can 
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be  adjusted  downward  to  preserve  the  demographic  balance  without  reducing  average  performance 
levels. 
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Proposed  New  Army  Aptitude  Area  Composites 


The  Army  Research  Institute  (ARI)  has  been  conducting  research  with  the  objective  of 
improving  the  effectiveness  of  Army  classification  (i.e.,  matching  of  recruits  to  initial  job  training 
assignments).  Under  current  Army  procedures,  applicants  are  offered  initial  training  in  MOS  for 
which  they  are  eligible  and  which  meet  the  accession  needs  of  the  Army.  Eligibility  is  based  on 
minimum  standards,  with  applicants’  AA  (Aptitude  Area)  composite  scores  serving  as  the  scale  upon 
which  MOS  eligibility  is  measured.  Two  major  recommendations  have  come  out  of  the  research 
conducted  by  ARI.  The  first  recommendation  is  to  improve  the  classification  process.  The  existing 
training  reservation  system  (known  as  REQUEST)  should  be  enhanced  so  that  recruits  are  assigned  to 
jobs  that  they  are  likely  to  perform  best,  while  meeting  the  training  management  goals  of  the  Army. 

The  Enlisted  Personnel  Allocation  System  enhancement,  now  under  development,  is  designed  to  push 
REQUEST  toward  optimized  assignments.  The  second  recommendation  is  to  improve  the 
classification  metric  by  replacing  the  AA  composites  with  more  precise  measures  of  predicted 
performance.  We  have  found  that  the  AA  composites  were  not  adequately  designed  for  today’s 
classification  task  of  distinguishing  recruit  predicted  performance  across  a  variety  of  jobs. 

In  this  report  we  summarize  the  development  of  new  classification-efficient  composites  and 
corresponding  job  families. 

Role  of  Test  Composites  in  Classification  and  Training  Management 

In  conducting  classification  and  training  management  functions,  the  Services  have  created 
occupational  aptitude  test  composites  derived  from  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB).  Each  Service  creates  and  applies  its  own  aptitude  composites  in  the  manner  that  it  sees  fit. 
Generally,  the  composites  are  chosen  and  applied  because  they  have  been  shown  to  predict  training 
performance  for  broad  classes  of  occupations  within  a  Service  (Eitelberg,  1988,  p.  69).  Thus,  for 
purposes  of  classification,  broad  classes  of  occupations  have  become  defined  with  respect  to  ASVAB 
composite  subject  areas.  We  will  refer  to  these  broad  classes  of  occupations  as  job  families. 

The  Army’s  current  nine  AA  composites  go  back  to  1976.  These  composites  were  formulated 
to  represent  measures  of  the  aptitudes  /  skills  required  for  training  and  assignment  to  the  corresponding 
nine  Army  job  families:  clerical,  combat,  electronics  repair,  field  artillery,  general  maintenance, 
mechanical  maintenance,  operators  /  food,  surveillance  /  communications,  and  skilled  technical.  Table 
1  depicts  the  AA  composites  and  their  component  ASVAB  subtests.  Eligibility  for  training  in  a 
particular  MOS  is  determined  by  the  recruit’s  AA  composite  score  relative  to  the  minimum  or  cutoff 
score  required  for  training  in  that  MOS.  For  example,  MOS  55B  (ammunitions  specialist)  requires  a 
minimum  score  of  95  on  the  General  Maintenance  composite  to  be  eligible  for  job  training  and 
assignment. 
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Table  1:  ASVAB  Subtests  Comprising  the  Army’s  AA  Composites  and  AFQT 


ASYA 

B  SUE 

1TESTS 

AR 

MK 

VE 

AS 

El 

GS 

MC 

CS 

NO 

AA  COMPOSITES 

Electronics  Repair 

X 

X 

X 

X 

General  Maintenance 

X 

X 

X 

X 

Mechanical  Maintenance 

X 

X 

X 

X 

Operators  /  Food 

X 

X 

X 

X 

Surveillance  /  Communications 

X 

X 

X 

Combat 

X 

X 

Field  Artillery 

X 

X 

Skilled  Technical 

X 

X 

X 

X 

Clerical 

X 

X 

X 

General  Technical 

X 

X 

AFQT 

X 

X 

XX 

ASVAB  is  comprised  of  following  subtests:  Arithmetic  Reasoning  (AR),  Math  Knowledge  (MK),  Verbal  (VE) 
=  Paragraph  Comprehension  (PC)  +  Word  Knowledge  (WK),  Auto  &  Shop  Information  (AS),  Electronics 
Information  (El),  General  Science  (GS),  Mechanical  Comprehension  (MC),  Coding  Speed  (CS),  Numerical 
Operations  (NO). 

The  classification  task  is  the  assignment  of  recruits  into  initial  job  training  in  order  to  provide 
the  best  match  or  fit  while  meeting  accession  and  training  management  goals.  While  the  formulation  of 
the  AA  composites  as  shown  in  the  table  has  a  certain  intuitive  appeal,  it  does  not  provide  the  requisite 
classification  capability. 

Fit  pertains  to  congruence  between  recruit  aptitudes  and  job  requirements,  and  should  be 
measured  with  predicted  performance  in  the  assigned  job  as  the  criterion.  The  AA  composites  were 
not  developed  with  a  job  performance  criterion.  The  training  criteria  used  to  validate  the  AA 
composites  were  the  best  available  criteria  at  the  time,  but  do  not  adequately  represent  soldier 
performance.1  Moreover,  each  composite  is  formulated  as  a  combination  of  4  unit- weighted  subtests 
(except  for  Clerical).  This  approach  is  a  carry-over  from  the  1950s  when  calculations  were  kept  as 
simple  as  possible,  reflecting  earlier  limitations  of  computing  power.  One  consequence  is  that  AA 
composites  do  not  track  performance  as  well  as  they  might.2  A  better  approach  is  full  least-squares 
estimation,  in  which  all  subtests  contribute  to  each  composite  in  proportion  to  their  power  to  explain 
the  variation  in  performance. 

Classification  efficiency  requires  the  use  of  predictors  that  can  distinguish  how  well  a  recruit  is 
likely  to  perform  in  different  jobs.  The  A  A  composites  have  limited  ability  to  do  this.  Consider  that 


1  In  the  mid-1980s  representative  MOS  were  examined  in  a  validation  of  the  AA  composites  using  school  grades,  early 
Skill  Qualification  Test  (SQT)  performance  data,  and  Project  A  criterion  measures,  though  the  unit-weighted  calculation 
procedures  were  retained  (see  Campbell  (1989)). 

2  Predictive  validity  estimates  in  the  literature  reveal  correlations  in  the  0.55  to  0.60  range  between  the  composites  and 
measures  of  training  and  job  performance. 
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five  pairs  (out  of  36  possible  pairs)  of  AA  composites  have  three  out  of  four  ASVAB  subtests  in 
common.  See  Table  1 :  these  are  electronics  repair  and  general  maintenance;  operators  /  food  and 
surveillance  /  communication;  operators  /  food  and  mechanical  maintenance;  surveillance  / 
communication  and  combat;  field  artillery  and  combat.  And  1 9  pairs  of  AA  composites  have  two  out 
of  four  ASVAB  subtests  in  common.  The  consequence  is  difficulty  in  distinguishing  predicted  recruit 
performance  across  these  pairs  of  job  families.3 

Developing  Classification-Efficient  Composites  and  Job  Families 

The  ASVAB  subtests,  which  underlie  the  AA  composites,  do  better  in  tracking  job 
performance.  Indeed,  ARI  Project  A  research  of  the  1980’s  found  a  relatively  strong  relationship 
between  ASVAB  factors  and  first-term  soldier  performance  measures  and  validated  the  use  of  ASVAB 
as  a  selection  tool.4  The  AFQT  composite  is  used  for  selection  into  the  services. 

With  the  success  of  the  ASVAB  factors  in  tracking  performance,  ARI  conducted  research 
beginning  in  the  early  1990’s  to  develop  classification-efficient  composites  for  use  in  Army 
classification  —  that  is,  to  develop  better  measures  for  distinguishing  recruit  capabilities  to  do  different 
jobs  and  to  test  the  efficacy  of  these  measures  for  making  better  classification  decisions.  This  research 
was  founded  upon  differential  assignment  theory  (DAT),  enhanced  and  applied  by  Dr.  Joseph  Zeidner 
and  Mr.  Cecil  Johnson  of  George  Washington  University.5 

This  research  has  grown  out  of  the  ongoing  debate  among  psychologists  about  the  nature  of 
intelligence  and  how  it  is  measured.  Generalizability  theory  suggests  that  one  general  ability  factor  is 
sufficient  to  predict  performance  on  jobs  with  a  strong  cognitive  component.  Proponents  of  the  theory 
believe  that  the  underlying  general  ability  factor  is  the  only  stable  predictor  of  job  performance  as 
measured  in  independent  samples.  If  it  is  true  that  specific  aptitudes  do  not  add  to  the  prediction  of  job 
performance,  then  more  efficient  classification  of  individuals  to  jobs  based  on  specific  aptitudes  is  not 
achievable.  However,  if  there  are  two  or  more  common  or  unique  factors  that  differentially  predict 
performance  in  various  jobs,  then  classification  efficiency  is  a  relevant  issue  (Zeidner  and  Johnson, 
1993,  p.377). 

In  developing  DAT  the  research  team  members  postulate  that  several  factors  differentially 
predict  performance  in  various  jobs,  and  argue  that  DAT  provides  a  more  coherent  framework  for  job 
classification,  although  recognizing  the  general  ability  factor  as  the  dominant  predictor  of  performance 
in  a  selection  model  (Zeidner  and  Johnson,  1993,  p.378).  The  empirical  implementation  of  DAT  has 
shown  that  there  are  useful  factors  contributing  a  nontrivial  amount  of  classification  efficiency  in 
addition  to  that  contributed  by  the  general  ability  factor.  This  includes  both  the  three  common  factors 


3  Additional  evidence  for  the  absence  of  differential  classification  power  can  be  seen  by  examining,  for  each  composite  or 
job  family,  how  many  times  other  composites  do  as  well  or  better  than  its  own  composite  in  explaining  performance.  The 
counts  are  as  follows:  CL,  7;  SC,  7;  FA,  5;  EL,  4;  GM,  4;  MM,  3;  OF,  3;  CO,  2;  and  ST,  0.  For  example,  at  one  extreme,  7 
of  the  9  composites  do  as  good  or  better  at  explaining  CL  performance  than  does  the  CL  composite  itself.  Validity  data 
source:  McLaughlin,  Rossmeissl,  et  al  (1984).  Note  that  data  refer  to  CL  and  SC  validities  before  they  were  revised  in 
1984. 

4  Four  ASVAB  factors  were  derived  using  factor  analysis:  verbal,  quantitative,  technical  knowledge,  and  perceptual 
speed.  This  is  summarized  in  Zook  (1996). 

5  Zeidner  and  Johnson  have  built  on  the  earlier  work  by  Brogden  (1946,  1959)  and  Horst  (1954). 
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and  several  of  the  unique  factors  found  in  the  ASYAB.  The  empirical  implementation  is  based  upon 
the  following  three  principles: 

•  One:  New  composites  should  utilize  defensible  criterion  data  (i.e.,  soldier  performance  data)  and 
all  the  informational  power  of  the  ASVAB  battery. 

In  the  estimation  and  testing  of  new  composites,  the  research  team  created  and  validated  a  job 
performance  database  of  260,000  Skill  Qualification  Test  (SQT)  observations  over  the  1987-89  period. 
A  carefully  developed  composite  of  hands-on  and  written  job  sample  tests,  known  as  the  Project  A  job- 
specific  technical  proficiency  measure,  was  used  to  validate  the  SQT  criterion.  The  researchers  found 
relationships  between  ASVAB  and  SQT  similar  to  those  found  between  ASVAB  and  Project  A 
measures,  providing  additional  confidence  in  reliance  upon  ASVAB  in  the  development  of  the  new 
composites  (Zeidner,  Johnson,  Vladimirsky,  1997). 

For  each  job  family,  new  composites  were  developed  by  estimating  the  functional  relationship 
between  individual  SQT  performance  and  ASVAB  subtest  scores  —  hence  the  name  predicted 
performance  composites.  The  resulting  composites  have  good  predictive  power  or  what  psychologists 
call  predictive  validity,  and  all  subtests  contribute  to  each  composite,  but  not  equally.  In  other  words, 
the  new  composites  utilize  weights  that  allow  each  subtest  to  make  its  maximum  contribution  (taking 
into  account  the  other  subtests  in  the  battery)  to  predicted  performance.  This  is  in  marked  contrast  to 
the  use  of  unit  weights  in  the  existing  composites,  in  which  the  contributions  of  selected  subtests  are 
predetermined  and  set  equal  to  each  other  without  regard  to  explanatory  power. 

•  Two:  Classification  efficiency  not  only  depends  upon  (a)  how  well  the  composites  predict 
performance  but  how  distinctive  the  composites  are,  one  from  another,  and  also  (b)  how  many 
distinct  job  families  can  be  identified  using  available  criterion  data.6  In  other  words,  predictive 
validity  is  only  one  term  in  the  classification  efficiency  equation  and,  thus,  classification  efficiency 
cannot  be  described  adequately  by  predictive  validity  alone. 

According  to  this  formulation,  predictive  validity  and  the  distinctiveness  of  the  composites  (as 
described  by  their  intercorrelations)  are  both  affected  by  increasing  the  number  of  job  families. 
Increasing  the  number  of  job  families  affects  validity  because  it  results  in  more  homogeneous  jobs 
being  placed  together  to  be  predicted  by  a  single  composite.  Increasing  the  number  of  job  families 
reduces  the  intercorrelations  among  the  composites  because  it  results  in  a  greater  uniqueness  in  the  job 
families.  And  finally,  as  the  number  of  job  families  increases,  it  is  possible  to  more  precisely  assign 
individuals  to  job  families  by  capitalizing  on  intra-individual  differences.  Using  the  SQT  database  and 
a  clustering  algorithm  designed  to  identify  classification-efficient  job  families,  two  sets  of 
classification-efficient  job  families  were  identified  -  a  detailed  set  of  150  and  a  summary  set  of  17  job 
families.7 


6  See  Brogden  (1959);  Johnson,  Zeidner,  and  Leaman  (1992,  p.  6). 

7  We  begin  with  a  set  of  1 50  job  families  that  can  be  identified  with  the  SQT  data  set.  In  this  initial  position,  maximum 
classification  efficiency  would  obtain  with  each  MOS  as  its  own  job  family.  The  clustering  procedure  entails  finding  the 
two  MOS  which  can  be  combined  into  a  single  job  family  with  the  smallest  loss  in  efficiency.  This  is  determined  with 
simulation  experiments,  and  the  selection  of  that  combination  which  yields  the  largest  MPP  of  assigned  recruits.  The  steps 
are  repeated  until  further  clustering  of  job  families  is  judged  to  reduce  classification  efficiency  by  an  unacceptable  amount. 
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We  can  illustrate  how  the  new  (classification-efficient)  composites  differ  from  the  existing  AA 
composites.  In  the  earlier  discussion  we  pointed  out  the  difficulty  of  distinguishing  between  the 
existing  Operators  /  Food  and  Surveillance  /  Communications  composites  because  they  share  three 
ASVAB  subtests  in  common  (AS,  MC,  VE).  However,  when  we  look  at  the  new  composites  (see 
Charts  1 A  and  IB)  we  see  notable  differences.8  In  the  first  place,  the  four  top  contributors  in  new 
Operators  /  Food  are  AS,  AR,  MC  and  VE;  now  AS,  MC,  and  VE  are  the  core  of  the  existing 
composite,  while  AR  has  a  very  important  role  in  new  Operators  /  Food  but  is  not  part  of  the  existing 
operators  composite.  Second,  the  four  top  contributors  in  Surveillance  /  Communications  are  MK,  VE, 
AR,  and  El;  only  AR  and  VE  are  part  of  the  existing  composite.  Indeed,  MK  plays  a  defining  role  in 
the  new  Surveillance  /  Communications  composite  but  is  not  part  of  the  existing  composite. 

•  Three:  Classification  efficiency  should  be  evaluated  by  measuring  soldier  mean  predicted 
performance  (MPP)  in  job  assignments  made  within  classification  optimization  experiments. 

The  simulation  experiments  utilize  Army  job  performance  and  ASVAB  data  circa  1989: 
individuals  are  assigned  so  as  to  maximize  predicted  performance  while  meeting  MOS  quotas  and 
certain  other  distributional  constraints  (but  stopping  short  of  the  complete  set  of  training  management 
constraints).  In  these  experimental  simulations  assignments  are  made  on  an  entity-by-entity  basis 
while  avoiding  the  assignment  of  an  entity  to  any  job  family  or  MOS  for  which  the  quotas  have  been 
filled  within  a  batch  or  sub-sample.  The  predicted  performance  of  each  entity  in  his/her  assigned  job 
family  is  computed,  and  the  average  or  mean  predicted  performance  is  computed. 

The  experiments  were  conducted  using  a  “triple  cross  validation”  design  to  assure  unbiased 
estimates.  The  design  requires  three  independent  samples:  (a)  an  analysis  sample,  for  estimating  the 
predicted  performance  function  used  during  the  assignment  process;  (b)  cross  samples  which  serve  as 
the  source  of  entities  that  are  optimally  assigned  in  the  simulation;  and  (c)  an  evaluation  sample,  for 
estimating  the  predicted  performance  function  used  in  the  evaluation  of  the  assignments  made.  Each 
experiment  is  conducted  twice,  with  the  role  of  analysis  and  evaluation  samples  reversed,  and  the 
results  averaged. 

The  results  of  the  classification  simulation  experiments  are  expressed  in  MPP  and  shown  in 
Table  2.  By  comparing  the  10  job  family9  new  composite  results  (MPP  =  .123)  with  the  existing  AA 
family  results  (MPP  =  .023)  we  see  that  the  new  composites  provide  more  than  five  times  the 
classification  power  as  measured  by  MPP.10  In  other  words,  improved  accuracy  in  the  estimation  of 
predicted  performance,  utilizing  only  the  existing  job  families,  promises  large  gains  in  classification. 
And  when  150  job  families  are  identified,  with  the  greater  homogeneity  of  these  families  and  greater 
overall  distinction  between  them,  the  classification  gains  increase  to  more  than  eight  times.  Indeed,  we 
estimate  that  the  performance  gains  obtainable  from  improved  classification  would  rival  in  size  those 
obtained  from  existing  selection-for-service.11 


8  This  illustration  is  taken  from  a  17  job  family  structure  proposed  as  the  second  tier  and  described  below. 

9  A  very  small  General  Technical  (GT)  family  is  included;  otherwise,  structure  is  same  as  existing  operational  families. 

10  Random  assignment  would  result  in  an  MPP  of  zero. 

11  That  is,  the  performance  gains  due  to  improved  classification  are  estimated  to  exceed  those  obtained  from  the  screening 
out  of  low  AFQT  applicants.  See  Zeidner,  Johnson,  Vladimirsky,  and  Weldon  (1997). 
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Chart  1A.  Existing  AA  Composites:  OF  vs.  SC 
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Chart  IB.  New  Composites:  OF  vs  SC 
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The  results  depicted  in  Table  2  (with  the  exception  of  the  last  row)  are  based  on  a  9-subtest 
ASVAB  battery  -  they  precede  the  DoD  decision  to  delete  the  Numerical  Operations  and  Coding 
Speed  subtests  from  the  battery.  The  last  row  indicates  the  overall  loss  in  classification  efficiency 
from  deletion  of  these  subtests.  We  would  expect  that  the  MPP  estimate  of  a  9-LSE  composite  based 
on  a  7-subtest  battery  would  be  reduced  in  the  same  proportion. 

In  the  simulation  experiments  described,  the  recruit  is  assumed  to  choose  the  job  opportunity  at 
the  top  of  his/her  list.  In  reality,  classification  gains  are  affected  by  the  behavior  of  recruits  choosing 
among  an  ordered-list  of  job  training  opportunities.  To  examine  the  effects  of  job  choice  behavior 
upon  classification  gains,  the  research  team  conducted  an  additional  set  of  simulation  experiments.12 
They  compared  choosing  from  the  top-of-the-list  job  versus  a  75%  probability  of  choosing  one  of  the 
top  five  jobs  on  the  ordered  list  —  deemed  to  be  a  reasonable  depiction  of  job  choice  behavior  in  an 
operational  setting  —  and  found  a  reduction  of  .072  in  MPP.  In  this  case,  the  classification  gains 
obtained  (i.e.,  MPP  =  .121)  are  still  more  than  five  times  those  achievable  using  optimization 
procedures  with  the  existing  AA  composites.13 


Table  2:  Estimated  MPP  -  Results  of  Simulation  Experiments 


Family  size 

Unbiased  MPP 

AA  -  9  operational  families  (unit  weights) 

.023 

10  (LSE  weights) 

.123 

13  (LSE  weights) 

.138 

17  (LSE  weights) 

.145 

150  (LSE  weights) 

.195 

150  (LSE  weights;  7  ASVAB  subtests) 

.183 

Source:  Zeidner,  Johnson,  Vladimirsky,  and  Weldon  (2000a,  Table  3)  and  (2000b,  Table  3). 


12  These  experiments  are  described  in  Johnson,  Zeidner,  Vladimirsky,  and  Weldon  (1999). 

13  The  monetary  value  of  these  classification  gains  can  be  estimated  as  (what  economists  call)  the  opportunity  cost  of 
retaining  the  current  AA  system.  In  the  present  context  of  an  optimizing  assignment  procedure,  this  is  the  additional  cost  of 
using  current  AA  composites  to  achieve  the  same  level  of  performance  gains  obtainable  using  the  new  composites. 
Specifically,  using  current  AA  composites,  how  many  additional  1-3  A  recruits,  in  place  of  3B  recruits,  would  be  required 
to  achieve  the  same  gains  obtained  using  the  new  composites,  and  what  would  it  cost  to  acquire  them? 

The  heart  of  the  opportunity  cost  calculation  is  determination  of  the  number  of  additional  1-3  A  recruits  required. 
Using  the  1997  accession  cohort  as  a  baseline,  the  assignments  made  using  the  current  procedures  are  ordered  from  high  to 
low  by  AFQT  score.  For  individuals  at  each  percentile  score,  average  and  cumulative  average  predicted  performance 
scores  for  the  job  assignments  actually  made  are  calculated.  To  meet  a  predetermined  overall  average  performance  target, 
individuals  from  the  bottom  are  successively  deleted  and  replaced  with  1-3A  recruits  (assumed  to  score  at  the  original  1-3A 
average)  until  the  performance  target  is  reached.  We  fmd  that  that  the  percentage  of  1-3A  recruits  would  have  to  rise  10  - 
15  percentage  points  and  would  conservatively  cost  in  the  neighborhood  of  $100  million  in  additional  recruiting  costs. 

Calculations  are  made  for  a  cohort  size  of  72,000,  with  1-3  A  recruits  comprising  about  68%.  Average  recruiting 
costs  are  $1 1,660  for  high-quality  and  $6,223  for  low-quality  recruits.  Marginal  costs  are  estimated  at  $35,555  for  high- 
quality  recruits,  and  assumed  to  increase  with  high-quality  share  (each  one  percent  increase  in  share  is  associated  with  a  one 
percent  increase  in  marginal  costs).  For  example,  at  80%  high-quality  share,  the  average  cost  has  increased  to  $14,935  for 
high-quality  recruits.  Recruiting  costs  refer  to  1995  (Source:  USACEAC  Army  Manpower  Cost  System). 
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Fairness  of  the  New  Composites 


As  part  of  the  development  of  the  new  composites,  the  ARI  research  team  has  also  examined 
the  fairness  of  the  new  composites  from  a  gender  and  race  perspective  (Zeidner,  Johnson,  Vladimirsky, 
1998).  Fairness  is  traditionally  defined  as  the  absence  of  under-predictions  for  minority  groups  for 
which  discrimination  potentially  exists.  In  other  words,  a  situation  in  which  the  predicted  performance 
of  minority  group  members  falls  below  their  actual  performance  would  call  into  question  the  fairness 
of  the  prediction  instrument  —  in  this  case  the  test  composites. 

The  fairness  of  the  new  vis-a-vis  the  existing  AA  composites  was  examined  using  data  based 
on  actual  assignments  of  recruits  and  data  from  simulation  experiments.  The  sample  in  both 
experiments  used  ARI’s  data  set  of  ASVAB  subtest  and  SQT  scores.14  In  the  actual  assignments 
experiment,  a  distinct  trend  of  under-predictions  was  found  for  females  and  blacks  for  both  existing 
and  new  composites.  While  the  prediction  errors  for  females  were  statistically  significant,  the  errors 
for  females  and  blacks  were  too  small  to  have  practical  significance.  In  the  simulation  experiments, 
again  there  was  a  consistent  trend  of  under-predictions  for  minorities,  the  new  vis-a-vis  existing 
composites  resulted  in  smaller  errors,  and  the  small  under-predictions  were  considered  of  little 
practical  consequence.  In  sum,  the  new  composites  are  at  least  as  fair  as  the  existing  composites.15 

Recommended  New  Composites  and  Job  Family  Structure 

The  classification  research  findings  suggest  that  new  (predicted  performance)  composites  are 
superior  to  the  existing  unit-weighted  composites.  Furthermore,  adoption  of  new  composites  and 
corresponding  job  families  within  an  enhanced  REQUEST  system  holds  the  promise  of  dramatic 
increases  in  soldier  performance.  A  two-tier  structure  is  recommended  for  operational  use. 

A  visible  tier  is  comprised  of  17  operational  families,  and  would  be  used  for  administrative, 
counseling,  and  school  proponent  purposes  just  as  the  existing  nine  operational  job  families  are  used. 
(An  invisible  tier  is  described  on  page  9.)  Minimum  eligibility  standards  (cutoff  scores)  would  be 
established  against  these  new  composites.  The  visible  tier  is  consistent  with  the  CMFs  currently  used 
by  the  Army  in  managing  the  entry-level  job  structure  and  is  also  consistent  with  the  current  AA 
system.  No  pair  of  MOS  that  are  together  in  the  current  AA  system  fails  to  be  together  in  the  17  job 
family  system.  The  visible  tier  strongly  resembles  the  existing  structure,  in  effect  being  a  further 
shredding  of  certain  existing  families.  Four  of  the  nine  existing  job  families  are  divided  into  two 
relatively  homogeneous  sets  of  clusters,  two  are  divided  into  three  sets  of  clusters,  and  three  remain  as 
single  clusters.  Provisional  MOS  membership  in  the  set  of  17  job  families  is  shown  in  Appendix  B. 
Note  that  recently  created  MOS  remain  to  be  inserted  into  their  appropriate  job  family  set. 


14  The  sample  was  limited  to  soldiers  having  the  task-based  written  test,  skill  level  1  SQTs  for  66  MOS  obtained  during 
FY  1987-89.  The  sample  was  further  limited  to  enlistees  who  had  taken  the  ASVAB  in  its  current  format.  The  total  sample 
size  in  the  first  experiment  was  83,000  enlistees  and  the  size  of  the  sample  for  the  second  experiment  was  a  stratified 
random  sample  of  30,000  first-term  enlistees  drawn  from  the  larger  sample. 

15  Additional  research  into  fairness  issues  is  underway  in  response  to  recommendations  made  by  an  expert  panel  convened 
in  September  2001  to  review  ARI’s  classification  research. 
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Table  3  provides  a  short  occupational  description  of  the  17  job  families  and  highlights  the 
major  differences  between  the  existing  (see  Table  1)  and  new  composites.  Generalizing  for  a  moment, 
we  can  characterize  the  new  composites  by  the  addition  of  AR,  VE,  and  MK.  More  detail  is  provided 
in  Appendix  A.  In  that  appendix,  the  reader  will  find:  (a)  a  table  providing  the  new  composite 
validities  (i.e.,  multiple  correlation  with  the  underlying  SQT  performance  criterion)  along  with 
composite  inter-correlations;  and  (b)  charts  describing  the  new  composites.  Each  chart  depicts  the 

Table  3:  New  Composite  Changes  and  Job  Family  Descriptions 


Family 

Occupation  description 

Highlights  of  the  major  changes  in 
composition  of  the  New  Composites  vis-a- 
vis  existing  composites 

CL-1 

CMF  71  clerical 

VE,  MK  dominate 

CL-2 

Other  clerical  functions 

Similar  to  existing  clerical  family 

E 

Infantry 

Adds  MK  in  predominant  role 

E 

Combat  engineer;  armor  crew 

Adds  MK,  VE  in  predominant  roles 

Combat  equipment  operator  /  maintainer 

Adds  VE,  AS;  relative  balance 

EL-2 

Equipment  operator 

Adds  VE,  AS;  MK  predominates 

EL-3 

Equipment  repairer 

Adds  VE  in  predominant  role 

FA 

Cannon  fire 

Adds  VE,  AS 

GM-1 

Mechanic  /  repairer 

Adds  AR 

GM-2 

Various 

Adds  AR,  MC 

MM-1 

Land  vehicle  mechanic  /  repairer 

AS  predominates 

MM-2 

Helicopter  /  aircraft  repairer 

Adds  VE,  AR,  MK 

OF 

Various  operators;  food  service 

Adds  AR 

SC 

Radar  /  radio  transmission 

Adds  MK,  El 

ST-1 

Medical 

Adds  AR;  relative  balance 

ST-2 

Analyst 

Adds  AR;  VE  dominates 

ST-3 

Operations  /  various 

Adds  AR;  relative  balance 

ASVAB  subtests  comprising  the  existing  AA  and  new  composites,  and  allows  the  reader  to  compare 
the  subtest  weighting  structures  across  existing  and  new  composites.  As  an  illustration,  consider  the 
shredding  of  the  existing  mechanical  maintenance  (MM)  job  family  into  two  new  families:  land 
vehicle  (MM-1)  and  aircraft  vehicle  (MM-2)  maintenance  (also  shown  in  Chart  2).  The  existing  MM 
composite  is  defined  by  AS,  MC,  El,  and  NO.  The  predominant  contributor  to  MM-1  is  AS,  followed 
distantly  by  MC,  El,  and  MK.  The  major  contributors  to  MM-2  are  VE,  AS,  AR,  MK,  and  MC.  The 
latter  shows  more  balance  in  the  aptitudes  identified  as  necessary  to  perform  in  the  job  family,  and  as  a 
consequence  female  qualification  rates  across  a  range  of  cutoff  scores  are  noticeably  higher  for  MM-2 
than  for  the  existing  MM  composite  (see  Chart  2). 

The  predicted  performance  composites  are  shown  in  Table  4  for  the  17  job  families,  each  row 
depicting  relative  subtest  weights. 
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Table  4:  17  LSE  Composites  -  Subtest  Weights  (Relative) 


gs 

ar 

as 

mk 

me 

ei 

ve 

1  CL1 

0.000 

0.000 

0.000 

0.985 

0.095 

0.022 

1.000 

2  CL2 

0.000 

1.000 

0.261 

0.712 

0.269 

0.175 

0.797 

3  COI 

0.259 

0.401 

0.658 

1.000 

0.532 

0.262 

0.398 

4  C02 

0.428 

0.813 

0.893 

1.000 

0.731 

0.518 

0.809 

5  ELI 

0.056 

0.842 

1.000 

0.808 

0.507 

0.720 

0.997 

6  EL2 

0.235 

0.636 

0.680 

1.000 

0.474 

0.556 

0.675 

7  EL3 

0.102 

0.675 

0.206 

0.479 

0.213 

0.239 

1.000 

8  FA 

0.249 

0.715 

0.673 

1.000 

0.700 

0.297 

0.586 

9  GM1 

0.335 

1.000 

0.932 

0.614 

0.429 

0.684 

0.414 

10  GM2 

0.461 

0.593 

1.000 

0.923 

0.545 

0.427 

0.390 

11  MM1 

0.065 

0.279 

1.000 

0.239 

0.359 

0.344 

0.137 

12  MM2 

0.020 

0.789 

0.954 

0.662 

0.647 

0.299 

1.000 

13  OF 

0.251 

0.962 

1.000 

0.600 

0.636 

0.377 

0.714 

14  SC 

0.019 

0.685 

0.437 

1.000 

0.386 

0.551 

0.915 

15  ST1 

0.390 

0.886 

0.456 

0.752 

0.659 

0.325 

1.000 

16  ST2 

0.120 

0.673 

0.177 

0.604 

0.268 

0.118 

1.000 

17  ST3 

0.112 

0.670 

0.449 

0.741 

0.461 

0.265 

1.000 

Predicted  performance  composites  have  also  been  estimated  for  1 50  job  families.  These 
constitute  the  “invisible”  tier,  and  would  be  embedded  within  the  appropriate  REQUEST  module16  for 
classification  optimization.  Thus,  the  classification  gains  embodied  by  greater  delineation  of  job 
families  is  obtained  without  the  (visible)  accompanying  administrative  complexity.  In  sum, 
differential  assignment  theory  supports  the  use  of  a  larger  number  of  job  families,  having  more 
homogeneity  within  and  greater  heterogeneity  between  families,  with  stable  weights  producing  higher 
MPPs.  More  stable  families  of  this  type  produce  larger  mean  validity  and  lower  intercorrelations,  thus 
providing  greater  optimization  efficiency. 

Adoption  by  the  Army 

A  two-phase  adoption  and  implementation  process  is  envisioned.  The  purpose  of  the  first 
phase  is  to  educate  and  inform  Army  management  about  the  issues,  and  obtain  consensus  and  approval 
for  implementation  of  the  recommendation.  As  part  of  the  “educate  and  inform”  management,  this 
phase  could  include  a  one-day  workshop  sponsored  by  DCSPER  /  PERSCOM,  school  proponent  site 
visits,  and  briefings  to  Army  management  to  synthesize  the  issues  and  school  site  findings.  Issues  that 
would  be  addressed  at  school  proponents  include:  placement  of  MOS  within  new  17  operational  job 
family  structure;  determination  of  how  changes  in  composites  would  affect  composition  of  MOS 
eligible  applicant  pool;  and  derivation  of  new  cutoff  scores  under  new  composites  /  job  family 
structure.  In  the  second  phase,  the  new  composites  and  job  family  structure  would  be  installed  into 
Army  recruiting  and  training  management  procedures  with  the  coordination  and  assistance  of  the 
affected  agencies. 


16  For  example,  the  Enlisted  Personnel  Allocation  System  ( EPAS)  under  development  is  designed  to  push  REQUEST 
toward  optimal  assignments.  EPAS  field-testing  is  scheduled  for  FY  2001-03. 
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Interim  Composites  (beginning  January  2002) 


In  early  2001  responsible  Army  agencies  determined  that  the  changes  in  databases  and 
procedures  to  effect  a  change  to  17  job  families  and  composites  could  not  be  accomplished  in  time  to 
meet  the  DOD  deadline  of  December  3 1 , 200 1 .  Accordingly,  ARI  was  tasked  to  develop  an  interim  set 
of  composites  that  retained  the  existing  nine  operational  job  families  and  utilized  the  remaining 
ASVAB  subtests  after  deleting  NO  and  CS.  Given  the  strong  preference  for  predicted  performance 
composites  by  ARI  and  its  DCSPER  sponsor,  ARI  estimated  least-squares  composites  (shown  in  Table 
5)  for  the  existing  job  families  using  the  SQT  database  previously  mentioned.  The  interim  composite 
validities  (i.e.,  multiple  correlations  with  SQT  scores),  along  with  the  composite  inter-correlations,  are 
shown  in  Appendix  C.  The  existing  and  interim  composites  are  compared  in  the  charts  in  Appendix  C. 

Table  5:  Interim  AA  Composites  -  Subtest  Weights  (Relative) 


gs 

ar 

as 

mk 

me 

ei 

ve 

CL 

0.000 

1.000 

0.110 

0.767 

0.148 

0.110 

0.980 

CO 

0.313 

0.532 

0.733 

1.000 

0.595 

0.343 

0.529 

EL 

0.151 

0.818 

0.754 

0.890 

0.469 

0.598 

1.000 

FA 

0.249 

0.715 

0.673 

1.000 

0.700 

0.297 

0.586 

GM 

0.411 

0.828 

1.000 

0.794 

0.503 

0.577 

0.417 

MM 

0.060 

0.339 

1.000 

0.289 

0.394 

0.340 

0.237 

OF 

0.251 

0.962 

1.000 

0.600 

0.636 

0.377 

0.714 

SC 

0.019 

0.685 

0.437 

1.000 

0.386 

0.551 

0.915 

ST 

0.187 

0.727 

0.357 

0.697 

0.446 

0.230 

1.000 

With  the  exception  of  the  clerical  composite,  the  interim  composites  are  noticeably  different 
from  the  existing  composites.  Four  groupings  of  the  interim  composites  are  suggested  for  descriptive 
purposes.  The  CL,  EL,  SC,  and  ST  composites  are  constructed  around  verbal,  arithmetic  reasoning, 
and  math  knowledge  subtests;  the  CO  and  FA  composites  feature  math  knowledge,  with  auto  /  shop 
information,  arithmetic  reasoning,  and  mechanical  comprehension;  the  GM  and  OF  composites  feature 
auto  /  shop  information  and  arithmetic  reasoning;  and  MM  features  auto  /  shop  information.  Keep  in 
mind  that  all  subtests  are  used  in  each  composite,  and  the  contribution  of  two  or  three  of  the  lesser- 
weighted  subtests  will  equal  that  of  one  of  the  stronger  subtests.  Accordingly,  the  composites  are  more 
diverse  than  this  description  indicates. 

The  interim  composites  represent  a  predicted  performance  metric  with  (as  it  turns  out)  strong 
relationships  between  the  composites  and  AFQT;  its  application  leads  to  compositional  effects  that 
favor  qualification  rates  of  l-IIIA’s  relative  to  IIIB’s  and  IV’ s.  As  a  new  metric,  the  interim 
composites  require  (small)  changes  in  cutoff  score  levels  in  order  to  maintain  the  existing  demographic 
balance  in  qualification  rates.  Because  the  average  predicted  performance  of  those  qualifying  under 
the  interim  composites  is  somewhat  higher  than  under  the  existing  composites,  cutoff  score  levels  can 
be  adjusted  downward  (when  necessary)  to  preserve  the  demographic  balance  without  reducing 
performance  standards. 


12 


The  compositional  effects  can  be  illustrated  by  looking  at  the  Mechanical  Maintenance 
composite  in  the  neighborhood  of  a  cutoff  score  of  100.  Table  6  shows  the  percentage  qualifying  at 
the  various  line  scores  in  this  neighborhood  under  the  existing  and  interim  composite.  As  can  be  seen, 
the  percentage  qualifying  (male  and  female)  in  the  IIIB  and  IV  test  score  categories  is  somewhat  lower 
under  the  interim  vis-a-vis  the  existing  composite.  This  is  addressed  by  lowering  the  cutoff  score  for 
the  affected  MOS;  we  found  that  the  average  predicted  performance  of  the  qualifying  populations 
under  the  existing  composite  at  a  cutoff  score  of  100  and  the  interim  composite  at  a  cutoff  score  of  98 
is  approximately  equal;  thus,  on  performance  grounds  alone,  a  drop  of  two  points  is  appropriate.  The 
policy  decision  in  this  case  to  set  the  new  cutoff  score  at  97  represents  a  decision  to  be  inclusive  and 
stretches  slightly  what  can  be  justified  by  the  increased  performance. 


Table  6.  Percentage  Qualifying  -  Existing  vs.  Interim  Composites 
Mechanical  Maintenance 

MOS  =  14J,  45E,  45N,  63A  (M),  63D,  63E,  63G,  63N,  63S,  63Y,  67R,  67S,  68J,  88K,  88P 

Line  Score 


Male 

All 

1-3A 

3B 

IV 

Female 

All 

1-3  A 

3B 

iV 

MM 

100 

65.9 

80.4 

40.2 

25 

26.8 

37.8 

5.9 

2.6 

MM-9 

100 

64.3 

81.4 

33.8 

16.2 

24.4 

36.2 

2.0 

0.6 

99 

66.9 

83.7 

37.0 

18.8 

27.0 

39.9 

2.6 

0.9 

98 

69.4 

85.8 

40.5 

21.5 

29.9 

43.9 

3.4 

1.2 

97 

71.9 

87.8 

44.0 

24.2 

33.0 

48.0 

4.4 

1.3 

96 

74.3 

89.6 

47.5 

27.3 

36.1 

52.2 

5.7 

1.5 

Note:  MM  is  the  existing  composite,  based  on  4  subtests,  including  NO  subtest. 
MM-9  is  the  interim  composite,  based  on  7  subtests  after  deletion  of  NO  and  CS  from 
the  battery. 


It  is  also  of  interest  to  illustrate  the  mitigating  effect  played  by  the  interim  composites  on 
female  qualification  rates.  Consider  again  the  Mechanical  Maintenance  composite.  The  deletion  of 
NO  from  the  ASVAB  directly  affects  the  MM  composite  and  female  qualification  rates  for  that 
composite.  Table  7  shows  percentage-qualifying  rates  at  different  line  scores  using  the  existing  MM, 
MM_3SSS  (which  is  the  unit-weighted  MM  with  NO  excluded),  and  MM-9  composites.  In  the  first 
stage  -  the  four  subtest  versus  the  three  subtest  unit-weighted  comparison,  we  see  that  qualifying  rates 
for  males  and  especially  females  are  lower  overall.  At  a  cutoff  score  of  95,  the  female  percentage- 
qualifying  overall  rate  falls  from  45.6%  to  28.6%  with  the  deletion  of  the  NO  subtest.  In  the  second 
stage  -  the  three  subtest  unit-weighted  versus  the  least-squares  composite,  we  see  that  percentage- 
qualifying  rates  are  slightly  higher  for  males  and  noticeably  higher  for  females,  indicating  mitigation 
by  the  least-squares  composite  over  the  existing  composite  with  NO  deleted. 
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Table  7.  Percentage  Qualifying  Rates  for  Mechanical  Maintenance  Composites 


Line  Score 

Male 

All 

1-3A 

3B 

IV 

Female 

All 

1-3A 

3B 

IV 

MM 

105 

52.3 

67.5 

25.0 

12.9 

14.7 

21.5 

1.8 

0.3 

100 

65.9 

80.4 

40.2 

25.0 

26.8 

37.8 

5.9 

2.6 

95 

79.5 

90.6 

59.9 

45.9 

45.6 

59.9 

18.8 

10.7 

90 

88.8 

95.8 

76.5 

67.5 

65.2 

78.3 

40.6 

27.8 

MMJ3SSS 

105 

48.8 

61.7 

25.8 

14.7 

8.6 

12.6 

1.1 

0.6 

100 

61.4 

74.3 

38.5 

25.0 

16.5 

23.6 

3.1 

1.5 

95 

73.1 

84.6 

52.8 

40.2 

28.6 

39.2 

8.3 

3.9 

90 

83.1 

91.8 

67.7 

56.8 

44.2 

57.2 

19.6 

12.9 

MM~9 

105 

50.8 

67.9 

19.9 

7.3 

13.3 

20.1 

0.5 

0.3 

100 

64.3 

81.4 

33.8 

16.2 

24.4 

36.2 

2.0 

0.6 

95 

76.6 

91.2 

51.1 

30.2 

39.6 

56.6 

7.3 

1.9 

90 

86.8 

96.8 

69.7 

50.4 

58.3 

77.9 

21.3 

7.2 

Note:  MM  is  the  existing  composite,  based  on  4  subtests,  including  NO  subtest. 
MM_3SSS  is  the  existing  unit-weighted  composite,  but  excluding  NO  subtest. 

MM-9  is  the  interim  composite,  based  on  7  subtests  after  deletion  of  NO  and  CS  from 
the  battery. 


Estimation  of  the  New  /  Interim  Composite  Weights 

This  section  summarizes  how  to  obtain  the  AA  composite  weights  (referred  to  as  “u  and  k” 

values  in  the  source  research  reports,  and  incorporated  into  the  subtest  weights  shown  in  Tables  4  and 

5)  for  operational  use  in  the  applicant  Youth  Population.17  The  validity  coefficients  we  wish  to 

maximize  in  the  Youth  Population  actually  exist  only  in  doubly  restricted  MOS  samples  containing  the 

Skill  Qualifications  Test  (SQT)  criterion  in  the  1987  -  1989  research  data  set.  Appropriate  corrections 

have  to  be  made  to  these  restricted  validity  coefficients  to  obtain  unrestricted  validity  coefficients  that, 

if  subjected  to  restriction  in  range  effects,  would  equal  what  was  obtained  in  the  MOS  samples.  We 

also  have  to  estimate  what  the  criterion  standard  deviation  (SD)  would  have  to  be  in  the  unrestricted 

18 

population  to  yield  the  criterion  SDs  observed  in  the  MOS  samples. 

Our  values  of  u  and  k  used  in  conjunction  with  operational  ASVAB  subtest  scores  provide 
composite  test  scores  with  weights  that  maximize  the  correlation  coefficients  between  composite  test 


17  The  methodology  summarized  here  pertains  to  the  development  of  both  interim  and  new  (17  job  family)  composites. 

For  a  detailed  discussion  of  estimation  in  the  face  of  restriction  in  range,  see  Appendix  D. 

18  It  should  be  noted  that  whenever  validity  coefficients  are  mentioned,  we  are  assuming  that  these  coefficients  have  been 
corrected  for  attenuation  with  respect  to  criterion  unreliability.  Even  if  we  should  refer  to  an  uncorrected  validity  coefficient 
(for  restriction  in  range),  this  “uncorrected”  coefficient  has  been  corrected  for  attenuation. 
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scores  (when  all  weights  are  constrained  to  be  positive  or  zero)  and  aggregated  criterion  scores  that 
represent  the  MOS  of  each  specified  aptitude  area.  The  correlation  coefficient  to  be  maximized  is 
computed  using  restriction-in-range  corrections  that  make  the  coefficient  a  best  estimate  of  what  it 
would  have  to  be  for  the  measured  restriction  effects  to  have  the  effect  of  reducing  the  Youth 
Population  correlation  coefficient  to  the  magnitude  observed  in  the  restricted  MOS  sample.  The 
validity  coefficients  and  criterion  scores  aggregated  across  MOS  samples  to  represent  each  aptitude 
area  are  doubly  restricted  since  the  soldiers  in  these  MOS  samples  have  been  selected  into  the  Army 
and  classified  and  assigned  to  the  MOS  in  which  each  validity  is  computed.  Each  such  validity 
coefficient  is  corrected  to  provide  an  estimate  of  what  the  validity  coefficient  would  be  if  the  criterion 
were  perfectly  reliable. 

The  incorporation  of  restriction  in  range  corrections  into  procedures  for  assuring  that  best  weighted 
composites  have  an  expected  mean  of  100  and  standard  deviation  of  20  in  the  Youth  Population  can  be 
summarized  as  follows: 

1 .  Obtain  the  corrected  subtest  validity  coefficients  in  each  MOS  sample.  This  correction  is  for 
the  unreliability  of  the  criterion  and  for  the  effect  of  double  restriction  in  range  on  the 
predictors  and  criterion  variables  in  each  MOS  sample.  This  step  provides  the  subtest  validity 
coefficients  corrected  to  the  Youth  Population. 

2.  For  each  subtest  multiply  each  corrected  test  validity  by  the  density  of  that  MOS  and  then  sum 
these  products  across  the  set  of  MOSs  contained  in  each  job  family  to  provide  an  estimate  of 
each  subtest  validity  (against  a  hypothetical  job  family  criterion)  within  the  restricted  job 
family. 

3.  Use  these  job  family  subtest  validity  coefficients  and  the  inter-correlation  coefficients  among 
the  subtests  to  compute  all  positive  or  zero  regression  weights  that  are  appropriate  for  applying 
to  subtest  standard  scores  (SSSs)  to  provide  LSEs  of  the  criterion.  We  find  and  compute  those 
positive  Beta  weights  (i.e.,  B-weights)  that  maximize  the  prediction  of  the  criterion  (as 
measured  by  multiple  correlation  coefficients)  for  all  possible  subsets  of  B-weights. 

4.  Separately  for  each  job  family,  determine  expected  SDs  of  the  hypothetical  job  family  criterion 
variables  as  expected  in  the  Youth  Population.  Setting  the  SDs  of  the  subtest  scores  to  10,  and 
using  these  criterion  SDs,  compute  the  raw  score  regression  weights  (i.e.,  b-weights)  by 
converting  each  B-weight.  Note  that  the  resulting  b-weighted  composites  will  have  SDs  equal 
to  values  less  than  10  because  of  the  effects  of  the  positive  inter-correlation  coefficients  among 
the  subtests. 

5.  Transform  the  b-weights  into  u,k  values  that  provide  composite  scores  with  expected  SDs  equal 
to  20  and  means  equal  to  100  when  applied  to  operational  subtest  scores,  (a)  To  do  this  we 
multiply  each  b-weight  by  a  composite  multiplier  (CM)  that  will  convert  the  composite  to  have 
a  SD  of  20  without  considering  the  composite  mean,  (b)  In  the  next  step,  the  constant  is 
calculated  to  center  the  composite  at  100. 
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APPENDIX  A 

9  AA  Composites  vs.  17  LSE  Composites 


1 .  17  LSE  composite  validities 

2.  17  LSE  composite  correlations 

3.  Nine  bar  charts  comparing  existing  AA  composites  with  new  17  LSE 
composites 

a.  Existing  composites  shown  in  gray 

b.  New  composites  shown  in  black  and  white 

c.  Job  families  /  aptitude  areas  are  as  follows:  Clerical  (CL);  Combat 
(CO);  Electronics  Repair  (EL);  Field  Artillery  (FA);  General 
Maintenance  (GM);  Mechanical  Maintenance  (MM);  Operators  /  Food 
(OF);  Surveillance  /  Communications  (SC);  Skilled  Technical  (ST) 

d.  ASVAB  subtests  are  as  follows:  general  science  (gs);  arithmetic 
reasoning  (ar);  auto  &  shop  information  (as);  mathematics  knowledge 
(mk);  mechanical  comprehension  (me);  electronics  information  (ei); 
verbal  (ve);  coding  speed  (cs);  numerical  operations  (no) 


A-l 


PROPOSED  NEW  COMPOSITES:  17  LSE 


17  LSE  COMPOSITE  VALIDITIES 
CL1  CL2  COI 

C02 

ELI 

EL2 

EL3 

FA 

GM1 

GM2 

MM1 

MM2 

OF 

SC 

ST1 

ST2 

ST3 

0.713  0.654  0.525 

0.64 

0.675 

0.682 

0.718 

0.594 

0.767 

0.639 

0.762 

0.748 

0.657 

0.661 

0.607 

0.719 

0.72 

17  LSE  COMPOSITE  CORRELATIONS 

CL1  CL2  COI  C02 

ELI 

EL2 

EL3 

FA 

GM1 

GM2 

MM1 

MM2 

OF 

SC 

ST  1 

ST2 

ST3 

CL1 

1.000 

0.965 

0.930 

0.927 

0.921 

0.936 

0.970 

0.933 

0.898 

0.906 

0.799 

0.922 

0.908 

0.958 

0.949 

0.976 

0.958 

CL2 

0.965 

1.000 

0.980 

0.981 

0.978 

0.985 

0.995 

0.985 

0.970 

0.969 

0.897 

0.979 

0.975 

0.994 

0.991 

0.996 

0.994 

COI 

0.930 

0.980 

1.000 

0.998 

0.994 

0.998 

0.977 

0.999 

0.993 

0.997 

0.956 

0.993 

0.994 

0.992 

0.993 

0.975 

0.991 

C02 

0.927 

0.981 

0.998 

1.000 

0.998 

0.999 

0.982 

0.999 

0.997 

0.998 

0.961 

0.997 

0.998 

0.994 

0.996 

0.979 

0.994 

ELI 

0.921 

0.978 

0.994 

0.998 

1.000 

0.997 

0.981 

0.995 

0.997 

0.996 

0.966 

0.998 

0.998 

0.992 

0.993 

0.976 

0.992 

EL2 

0.936 

0.985 

0.998 

0.999 

0.997 

1.000 

0.985 

0.999 

0.995 

0.996 

0.953 

0.996 

0.995 

0.997 

0.996 

0.982 

0.995 

EL3 

0.970 

0.995 

0.977 

0.982 

0.981 

0.985 

1.000 

0.982 

0.970 

0.968 

0.901 

0.982 

0.976 

0.994 

0.994 

0.999 

0.996 

FA 

0.933 

0.985 

0.999 

0.999 

0.995 

0.999 

0.982 

1.000 

0.993 

0.996 

0.952 

0.995 

0.996 

0.995 

0.996 

0.981 

0.994 

GM1 

0.898 

0.970 

0.993 

0.997 

0.997 

0.995 

0.970 

0.993 

1.000 

0.998 

0.974 

0.995 

0.998 

0.986 

0.988 

0.964 

0.985 

GM2 

0.906 

0.969 

0.997 

0.998 

0.996 

0.996 

0.968 

0.996 

0.998 

1.000 

0.974 

0.995 

0.997 

0.986 

0.988 

0.964 

0.985 

MM1 

0.799 

0.897 

0.956 

0.961 

0.966 

0.953 

0.901 

0.952 

0.974 

0.974 

1.000 

0.965 

0.972 

0.929 

0.935 

0.889 

0.930 

MM2 

0.922 

0.979 

0.993 

0.997 

0.998 

0.996 

0.982 

0.995 

0.995 

0.995 

0.965 

1.000 

0.999 

0.991 

0.994 

0.977 

0.993 

OF 

0.908 

0.975 

0.994 

0.998 

0.998 

0.995 

0.976 

0.996 

0.998 

0.997 

0.972 

0.999 

1.000 

0.988 

0.992 

0.971 

0.989 

SC 

0.958 

0.994 

0.992 

0.994 

0.992 

0.997 

0.994 

0.995 

0.986 

0.986 

0.929 

0.991 

0.988 

1.000 

0.997 

0.992 

0.998 

ST1 

0.949 

0.991 

0.993 

0.996 

0.993 

0.996 

0.994 

0.996 

0.988 

0.988 

0.935 

0.994 

0.992 

0.997 

1.000 

0.992 

0.999 

ST2 

0.976 

0.996 

0.975 

0.979 

0.976 

0.982 

0.999 

0.981 

0.964 

0.964 

0.889 

0.977 

0.971 

0.992 

0.992 

1.000 

0.995 

ST3 

0.958 

0.994 

0.991 

0.994 

0.992 

0.995 

0.996 

0.994 

0.985 

0.985 

0.930 

0.993 

0.989 

0.998 

0.999 

0.995 

1.000 
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Clerical  Composites  --Existing  vs.  New 


Com  bat  Composites  -  Existing  vs.  New 


Subtest  relative  weights  Subtest  relative  weig  hts 


Electronics  Repair  Composites  -  Existing  vs.  New 
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General  Maintenance  Composites  -  Existing  vs.  New 


gs  ar  as  mk  me  ei  ve 

ASVAB  subtests 


E3GM<a<\A)  *GM-1(17)  DGM-2(17) 


Mechanical  Maintenance  Composite  -  Existing  vs.  New 


gs 


ASVAB  subtests 


Sub  test  relative  weights  Sub  test  relative  weights 


Operators  /  Food  Composite  -  Existing  vs.  New 
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Sub  test  relative  weights 


Skilled  Tech  nical  Composites  -■  Existing  vs.  New 
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APPENDIX  B 

MOS  Membership  in  17  Job  Family  Structure 

17  Job  Family  Structure: 

1  =  Clerical-1;  2  =  Clerical-2 

3  =  Combat-1;  4  =  Combat-2 

5  =  Electronics  Repair-1 ;  6  =  Electronics  Repair-2;  7  =  Electronics  Repair-3 

8  =  Field  Artillery 

9  =  General  Maintenance-1;  10  =  General  Maintenance-2 

11  =  Mechanical  Maintenance-1;  12  =  Mechanical  Maintenance-2 

13  =  Operators  /  Food 

14  =  Surveillance  /  Communications 

15  =  Skilled  Technical-1;  16  =  Skilled  Technical-2;  17  =  Skilled  Technical-3 

12/20/01 

Entry-level  MOS  List  [PERSCOM  11/2/2001] 

MOS 

TITLE 

**  17 

CMF 

Family 

56M 

Chaplain  assistant 

56 

1 

71D 

Legal  specialist  -  del  01 1 0 

71 

1 

71L 

Administrative  specialist 

71 

1 

73C 

Finance  specialist 

71 

1 

73D 

Accounting  specialist 

71 

1 

75B 

Personnel  administration  specialist 

71 

1 

75F 

Personnel  information  system  management  specialist 

71 

1 

75H 

Personnel  services  specialist 

71 

1 

88N 

Transportation  management  coordinator 

88 

1 

77F 

Petroleum  supply  specialist 

77 

2 

92A 

Automated  logistical  specialist 

92 

2 

92Y 

Unit  supply  specialist 

92 

2 

1 1 B 

Infantryman 

11 

3 

11C 

Indirect  fire  infantryman 

11 

3 

1 1 H 

Heavy  antiarmor  weapons  infantryman 

11 

3 

11M 

Fighting  vehicle  infantryman 

11 

3 

11X 

11 

3 

12B 

Combat  engineer 

12 

4 

12C 

Bridge  crewmember 

12 

4 

19D 

Cavalry  scout 

19 

4 

19K 

Ml  armor  crewman 

19 

4 

14E 

Patriot  fire  control  enhanced  operator/maintainer 

14 

5 

14L 

AN/TSQ-73  air  defense  artillery  command  and  control 
system  operator/maintainer  RC 

14 

5 

14M 

Man  portable  air  defense  system  crewmember  RC 

14 

5 

14T 

Patriot  launching  station  enhanced  operator/maintainer 

14 

5 

23R 

Hawk  missile  system  mechanic  -  del  0110  RC 

14 

5 

31L 

Cable  systems  installer/maintainer 

31 

5" 

51R 

Interior  electrician 

51 

5 

52G 

Transmission  and  distribution  specialist  RC 

51 

5 

B-l 


68X 

AH-64A  armament/electrical  systems  repairer 

67 

5 

13D 

Field  artillery  tactical  data  systems  specialist 

13 

6 

25R 

Visual  information  equipment  operator/maintainer 

25 

6 

25V 

Combat  documentation/production  specialist 

25 

6 

31F 

Network  switching  systems  operator/maintainer 

31 

6 

31P 

Microwave  systems  operator/maintainer 

31 

6 

31S 

Satellite  communication  systems  operator/maintainer 

31 

6 

31U 

Signal  support  systems  specialist 

31 

6 

35Y 

Integrated  family  of  test  equipment  operator/maintainer 

35 

6 

93F 

Field  artillery  meteorological  crewmember 

13 

6 

96H 

Imagery  ground  station  operator 

96 

6 

96  R 

Ground  surveillance  systems  operator 

96 

6 

27E 

Land  combat  electronic  missile  system  repairer 

35 

7 

27G 

Chaparral  and  Redeye  repairer 

35 

7 

27M 

MLRS  repairer 

35 

7 

27T 

Avenger  system  repairer 

35 

7 

35B 

LCSS  test  specialist 

35 

7 

35D 

Air  traffic  control  equipment  repairer 

35 

7 

35E 

Radio  and  communications  security  repairer 

35 

7 

35F 

Special  electronic  devices  repairer 

35 

7 

35H 

Test,  measurement,  and  diagnostic  equipment  maintenance 
support  specialist 

35 

7 

35J 

Computer/automation  system  repairer 

35 

7 

35L 

Avionic  communication  equipment  repairer 

35 

7 

35M 

Radar  repairer 

35 

7 

35N 

Wire  systems  equipment  repairer 

35 

7 

35R 

Avionics  systems  repairer 

35 

7 

39B 

Automatic  test  equipment  operator/maintainer 

35 

7 

45G 

Fire  control  repairer 

63 

7 

68J 

Aircraft  armament/missile  systems  repairer 

67 

7 

68N 

Avionic  /  mechanic 

67 

7 

68S 

OH-58D  armament/electrical/avionics  systems  repairer 

67 

7 

68Y 

AH-64D  armament/electrical/avionic  systems  repairer 

67 

7 

13B 

Cannon  crewmember 

13 

8 

13C 

Tactical  automated  fire  control  systems  specialist 

13 

8 

13E 

Cannon  fire  direction  specialist 

13 

8 

13F 

Fire  support  specialist 

13 

8 

13P 

MLRS  operations/fire  direction  specialist 

13 

8 

44  B 

Metal  worker 

63 

9 

44  E 

Machinist 

63 

9 

45B 

Small  arms/artillery  repairer 

63 

9 

45D 

Self-propelled  field  artillery  turret  mechanic 

63 

9 

45K 

Armament  repairer 

63 

9 

45T 

Bradley  fighting  vehicle  system  turret  mechanic 

63 

9 

52C 

Utilities  equipment  repairer 

63 

9 

52  D 

Power-generation  equipment  repairer 

63 

9 

43M 

Fabric  repair  specialist  -  del  0110 

92 

10 

51B 

Carpentry  and  masonry  specialist 

51 

10 

51K 

Plumber 

51 

10 

51M 

Firefighter 

51 

10 
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55B 

Ammunition  specialist 

55 

10 

55D 

Explosive  ordnance  disposal  specialist 

55 

10 

62E 

Heavy  construction  equipment  operator 

51 

10 

62F 

Crane  operator 

51 

10 

62G 

Quarrying  specialist  RC 

51 

10 

62  H 

Concrete  and  asphalt  equipment  operator 

51 

10 

62J 

General  construction  equipment  operator 

51 

10 

77W 

Water  treatment  specialist 

77 

10 

88H 

Cargo  specialist 

88 

10 

92M 

Mortuary  affairs  specialist 

92 

10 

92  R 

Parachute  rigger 

92 

10 

92S 

Laundry  and  textile  specialist 

92 

10 

45E 

Ml  Abrams  tank  turret  mechanic 

63 

11 

45N 

M60A1/A3  tank  turret  mechanic  RC 

63 

11 

62B 

Construction  equipment  repairer 

63 

11 

63A 

Ml  Abrams  tank  system  maintainer 

63 

11 

63B 

Light-wheel  vehicle  mechanic 

63 

11 

63D 

Self-propelled  field  artillery  system  mechanic 

63 

11 

63E 

Ml  Abrams  tank  system  mechanic 

63" 

11 

63G 

Fuel  and  electrical  systems  repairer 

63 

11 

63H 

Track  vehicle  repairer 

63 

11 

63J 

Quartermaster  and  chemical  equipment  repairer 

63 

11 

63M 

Bradley  fighting  vehicle  system  maintainer 

63 

11 

63N 

M60A1/A3  tank  system  mechanic  RC 

63 

11 

63S 

Heavy-wheel  vehicle  mechanic 

63 

11 

63T 

Bradley  fighting  vehicle  system  mechanic 

63 

11 

63W 

Wheel  vehicle  repairer 

63 

11 

63Y 

Track  vehicle  mechanic 

63 

11 

88K 

Watercraft  operator 

88 

11 

88L 

Watercraft  engineer 

88 

11 

88P 

Railway  equipment  repairer  RC 

88 

11 

88T 

Railway  section  repairer  RC 

88 

11 

88U 

Railway  operations  crewmember  RC 

88 

11 

67G 

Utility  airplane  repairer  RC 

67 

12 

67N 

UH-1  helicopter  repairer 

67 

12 

67R 

AH -64  attach  helicopter  repairer 

67 

12” 

67S 

OH-58D  helicopter  repairer 

67" 

12 

67T 

UH-60  helicopter  repairer 

67 

12 

67U 

CH-47  helicopter  repairer 

67 

12 

67V 

Observation/scout  helicopter  repairer 

67 

12 

67Y 

AH-1  attack  helicopter  repairer 

67 

12 

68B 

Aircraft  powerplant  repairer 

67 

12 

68D 

Aircraft  powertrain  repairer 

67 

12 

68F 

Aircraft  electrician 

67 

12 

68G 

Aircraft  structural  repairer 

67 

12 

68H 

Aircraft  pneudraulics  repairer 

67 

12 

13M 

Multiple  launch  rocket  system  crewmember 

13 

13 

14D 

Hawk  missile  system  crewmember  RC 

14 

13 

14J 

Air  defense  command,  control,  communications,  computers, 
and  intelligence  tactical  operations  center  enhanced 

14 

13 
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operator/maintainer 

14R 

Bradley  linebacker  crewmember 

14 

13 

14S 

Avenger  crewmember 

14 

13 

88M 

Motor  transport  operator 

88 

13 

91 M 

Hospital  food  service  specialist 

91 

13 

92G 

Food  service  specialist 

92 

13 

13R 

Field  artillery  firefinder  radar  operator 

13 

14 

31C 

Radio  operator-maintainer 

31 

14 

31R 

Multi-channel  transmission  systems  operator/maintainer 

31 

14 

74C 

Telecommunications  operator/maintainer 

74 

14 

77L 

Petroleum  laboratory  specialist 

77 

15 

91A 

Medical  equipment  repairer 

91 

15 

91 D 

Operating  room  specialist 

91 

15 

91E 

Dental  specialist 

91 

15 

91G 

Patient  administration  specialist 

91 

15 

91 H 

Optical  laboratory  specialist 

91 

15 

91 J 

Medical  logistics  specialist 

91 

15 

91K 

Medical  laboratory  specialist 

91 

15 

91 P 

Radiology  specialist 

91 

15 

91Q 

Pharmacy  specialist 

91 

15 

91R 

Veterinary  food  inspection  specialist 

91 

15 

91S 

Preventive  medicine  specialist 

91 

15 

91T 

Animal  care  specialist 

91 

15 

91X 

91 

15 

25M 

Multimedia  illustrator 

25 

16 

33W 

Intelligence  and  electronic  warfare  system  repairer 

33 

16 

37F 

Psychological  operations  specialist 

37 

16 

38A 

Civil  affairs  specialist  RC 

38 

16 

46Q 

Journalist 

46 

16 

46R 

Broadcast  journalist 

46 

16 

51T 

Technical  engineer 

51 

16 

74B 

Information  systems  operator/analyst 

74 

16 

81 L 

Lithographer 

81 

16 

81T 

Topographic  analyst 

81 

16 

82  D 

Topographic  surveyor 

81 

16 

96B 

Intelligence  analyst 

96 

16 

96D 

Imagery  analyst 

96 

16 

96U 

Unmanned  aerial  vehicle  operator 

96 

16 

97B 

Counter-intelligence  agent 

96 

16 

97L 

Translator/interpreter 

96 

16 

98C 

Signals  intelligence  analyst 

98 

16 

98H 

Communications  locator/interceptor 

98 

16 

98J 

Electronic  intelligence  interceptor/analyst 

98 

16 

98K 

Signal  collection/identification  analyst 

98 

16 

98X 

98 

16 

54  B 

Chemical  operations  specialist 

54 

17 

82C 

Field  artillery  surveyor 

13 

17 

93C 

Air  traffic  control  operator 

93 

17 

93P 

Aviation  operations  specialist 

93 

17 

95B 

Military  police 

95 

17 
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95C 

Internment/resettlement  specialist 

95 

17 

OOB 

Diver 

51 

13W 

Field  artillery  meteorological  crewmember  -  add  0304 

13 

13X 

Field  artillery  computer  systems  specialist 

13 

27D 

Paralegal  specialist 

27 

91Z 

Chief  medical  NCO 

91 
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APPENDIX  C 

9  AA  Composites  vs.  9  LSE  (Interim)  Composites 


1 .  9  LSE  composite  validities 

2.  9  LSE  composite  correlations 

3.  Nine  bar  charts  comparing  existing  9  AA  composites  with  interim 
9  LSE  composites 

a.  Existing  composites  shown  in  gray 

b.  Interim  composites  shown  in  black 

c.  Job  families  /  aptitude  areas  are  as  follows:  Clerical  (CL); 
Combat  (CO);  Electronics  Repair  (EL);  Field  Artillery  (FA); 
General  Maintenance  (GM);  Mechanical  Maintenance 
(MM);  Operators  /  Food  (OF);  Surveillance  / 
Communications  (SC);  Skilled  Technical  (ST) 

d.  ASVAB  subtests  are  as  follows:  general  science  (gs); 
arithmetic  reasoning  (ar);  auto  &  shop  information  (as); 
mathematics  knowledge  (ink);  mechanical  comprehension 
(me);  electronics  information  (ei);  verbal  (ve);  coding  speed 
(cs);  numerical  operations  (no) 


c-i 


INTERIM  COMPOSITES 

9-LSE  COMPOSITE  VALIDITIES  AND  CORRELATIONS 


9  LSE  COMPOSITE  VALIDITIES 


CL 

CO 

EL 

FA 

0.677 

0.566 

0.684 

0.594 

COMPOSITE  VALIDITIES 

CL 

CO 

EL 

FA 

CL 

1.000 

0.963 

0.973 

0.969 

CO 

0.963 

1.000 

0.997 

0.999 

EL 

0.973 

0.997 

1.000 

0.997 

FA 

0.969 

0.999 

0.997 

1.000 

GM 

0.948 

0.997 

0.994 

0.995 

MM 

0.883 

0.971 

0.965 

0.965 

OF 

0.954 

0.996 

0.996 

0.996 

SC 

0.984 

0.993 

0.997 

0.995 

ST 

0.988 

0.990 

0.996 

0.993 

GM 

MM 

OF 

SC 

ST 

0.698 

0.756 

0.657 

0.661 

0.679 

GM 

MM 

OF 

SC 

ST 

0.948 

0.883 

0.954 

0.984 

0.988 

0.997 

0.971 

0.996 

0.993 

0.990 

0.994 

0.965 

0.996 

0.997 

0.996 

0.995 

0.965 

0.996 

0.995 

0.993 

1.000 

0.983 

0.998 

0.986 

0.982 

0.983 

1.000 

0.982 

0.946 

0.939 

0.998 

0.982 

1.000 

0.988 

0.986 

0.986 

0.946 

0.988 

1.000 

0.998 

0.982 

0.939 

0.986 

0.998 

1.000 
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Clerical  Composites  ~  Existing  vs.  Interim 


gs  ar  as  mk  me  ei  ve  no  cs 

ASVAB  subtests 


□CU9M)  MCU9LSE) 


Combat  Composites  -  Existing  vs.  Interim 


gs  ar  as  mk  me  ei  ve  no  cs 

ASVAB  subtests 


0CO(9AA)  ■CO(9LSE) 
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Sub  test  relative  weights  Sub  test  relative  weights 


Electronics  Repair  Composites  -  Existing  vs.  Interim 


1.200 


1.000 


0.800 


0.600 


0.000 


mk  me 

ASVAB  subtests 


□  EU9AA)  ■EL(9LSE) 


Field  Artillery  Composites  —  Existing  vs.  Interim 


1.200 

1.000 

0.800 

0.600 

0.400 

0.200 

0.000 


gs  ar 


mk  me  ei  ve 

ASVAB  subtests 


no  cs 


□FA(9AA)  BFA(9LSE) 
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Mechanical  Maintenance  Composites  -  Existing  vs.  Interi 


Subtest  relative  weights 


Skilled  Technical  Composites  -  Existing  vs.  Interim 
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Appendix  D 

Restriction  in  Range  and  the  Estimation  of  AA  Composites 

Introduction 


This  appendix  will  focus  on  how  to  obtain  the  AA  composite  regression  weights 
(referred  to  as  “u  and  k”  values)  for  operational  use  in  the  applicant  Youth  Population.19 
The  validity  coefficients  we  wish  to  maximize  in  the  Youth  Population  actually  exist  only 
in  doubly  restricted  MOS  samples  containing  the  Skill  Qualifications  Test  (SQT) 
criterion  in  the  1987  -  1989  research  data  set.  Appropriate  corrections  have  to  be  made  to 
these  restricted  validity  coefficients  to  obtain  unrestricted  validity  coefficients  that,  if 
subjected  to  restriction  in  range  effects,  would  equal  what  was  obtained  in  the  MOS 
samples.  We  also  have  to  estimate  what  the  criterion  standard  deviation  (SD)  would  have 
to  be  in  the  unrestricted  population  to  yield  the  criterion  SDs  observed  in  the  MOS 
samples.20 

The  Army  operational  process  involves  an  applicant  Youth  Population  from 
which  self-selection  first  occurs,  and  then  the  Recruiting  Command  selects  some  and 
rejects  others  using  tests,  medical  examinations,  security  investigations  etc.  This  results 
in  an  Army  Input  Population  from  which  classification  and  assignment  procedures  and 
further  self  selection  create  the  150  MOS  samples,  each  with  its  separate  SQT  criterion 
measure.  Thus  there  is  a  selection  stage  and  a  classification  and  assignment  stage,  with  a 
restriction  in  range  effect  on  both  test  scores  and  hypothetical  criterion  scores  occurring 
at  both  stages.  If  we  confined  selection  effects  to  the  impact  of  the  AFQT  screen,  the 
selection  and  classification  stage  effects  would  have  to  be  corrected  in  a  sequential 
manner.  However,  since  we  are  not  restricting  ourselves  to  such  a  limited  selection 
effect,  and  are  instead  considering  all  effects  on  the  subtest  co-variances  at  each 
restriction  stage,  we  can  correct  validity  coefficients  and  criterion  SDs  directly  to  the 
Youth  Population. 

Our  correction  process  for  restriction  in  range  involves  contrasting,  separately  for 
each  MOS,  the  within-MOS  subtest  variance/co-variances  against  the  Youth  Population 
operational  test  variance/co-variances.  The  differences  in  the  variance/co-variances 
across  the  unrestricted  and  the  restricted  samples  for  variables  specified  as  explicitly 
selected  variables  are  the  measures  of  the  magnitude  of  the  restriction  effect.  For  our 
purposes  we  use  all  ASVAB  subtests  as  the  explicitly  restricted  variables  and  we 
designate  the  criterion  variables  as  the  implicitly  restricted  variables  that  are  restricted  to 
the  extent  that  they  are  predicted  by  the  explicitly  restricted  variables. 

Using  this  concept  we  can  calculate  the  effect  selection  has  on  subtest  scores  and 
can  then  calculate  the  further  effect  classification  and  assignment  has  on  test  scores  in  the 


19  This  appendix  has  been  prepared  by  Cecil  Johnson,  consulting  research  psychologist. 

20  It  should  be  noted  that  whenever  validity  coefficients  are  mentioned,  we  are  assuming  that  these 
coefficients  have  been  corrected  for  attenuation  with  respect  to  criterion  unreliability.  Even  if  we  should 
refer  to  an  uncorrected  validity  coefficient  (for  restriction  in  range),  this  “uncorrected”  coefficient  has  been 
corrected  for  attenuation. 
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Army  Input  Population  -  to  arrive  at  the  doubly  restricted  subtest  scores  in  the  MOS 
samples.  Considering  the  correlation  of  the  subtest  scores  with  the  criterion  scores  and 
the  amount  of  restriction  occurring  at  each  stage,  we  can  determine  the  restriction  effect 
on  the  hypothetical  criterion  scores  and  then  provide  a  correction  extending  from  the 
MOS  criterion  scores  to  the  less  restricted  populations  where  the  criterion  scores  exist 
only  as  a  function  of  the  subtest  scores  (i.e.,  as  predicted  criterion  scores). 

Approach 

There  is  more  than  one  algebraically  equivalent  way  of  providing  operational  u 
and  k  values  when  criterion  scores  are  only  available  on  the  doubly  restricted  MOS 
samples.  We  will  use  an  approach  that  utilizes  the  equality  of  G-weights  computed  in  the 
restricted  and  the  unrestricted  population  (using  Gulliksen’s  formulation  as  described 
below).  The  G-weights  computed  in  the  restricted  population  samples  will  be  used  as  a 
substitute  for  the  unobtainable  G-weights  in  the  unrestricted  population  in  Gulliksen’s 
formula  for  computing  the  criterion  variance  in  the  unrestricted  population. 

1 .  Consider  the  matrix  of  G-weights,  G,  in  each  MOS  sample.  Our  use  for  G  is  as  an 
entry  value  in  Gulliksen’s  formula  (see  below).  The  corrected  validity  coefficients, 
obtained  with  the  use  of  the  formula  at  either  or  both  the  Army  Input  Population  and 
Youth  Population  points,  were  then  employed  in  computing  Beta  weights  in  the 
Youth  Population.  Note  that  this  correction  must  be  made  from  each  MOS  sample  to 
the  Youth  population  to  produce  validity  coefficients  corrected  for  restriction  in 
range.  These  corrected  MOS  validity  coefficients  are  then  aggregated  into  a  corrected 
validity  for  each  specified  family,  using  acquisition  values  to  weight  the  MOS 
validity  coefficients  corrected  to  the  Youth  Population. 

2.  Visualize  a  composite  computed  for  an  individual  by  summing  the  product  of  each 
subtest  standard  score  (X)  and  B.  The  best  weighted  composite  XB  will  have  a  SD 
equal  to  the  validity  of  predicted  performance  (PP)  in  the  Youth  Population  if  the 
elements  of  the  V  matrix  used  in  computing  B  are  validity  coefficients  corrected  for 
restriction  in  range  to  represent  the  Youth  Population,  and  the  R  matrix  consists  of  the 
inter-correlation  coefficients  among  subtests  as  expected  in  the  Youth  Population. 

The  criterion  variables,  predicted  as  least  square  estimates  (LSEs)  by  the  PP 
composites,  have  a  SD  equal  to  1 .0  in  the  restricted  MOS  samples,  while  the 
hypothetical  unrestricted  criterion  variables  would  have  larger  SDs  in  the  less 
restricted  populations.  Compute  the  Youth  Population  beta  weights  as  follows: 

B  =  R1  VT, 

where  R  is  the  Youth  Population  matrix  of  subtest  inter-correlation  coefficients,  V  is 
the  matrix  of  validity  coefficients  corrected  to  the  Youth  Population,  and  superscript 
T  indicates  transposition  of  the  matrix.  Looking  at  the  formula  in  more  detail, 

R  =  Sx  Cxx  Sx,  and  VT  =  Sx  Cxc  SC; 

where  C  represents  criterion  /  subtest  variance  and  co-variances  found  in  Gulliksen’s 
formulae,  and  S  represents  a  diagonal  matrix  where  each  diagonal  element  is  equal  to 
a  reciprocal  of  a  SD. 
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3.  Compute  b-weights  by  converting  the  Beta  weights  computed  in  step  2.  The  b- 
weights  that  are  appropriate  to  apply  to  operational  test  scores  to  obtain  a  least 
squares  estimate  (LSE)  of  the  criterion  can  be  defined  in  terms  of  the  Beta  weights, 
the  SDs  of  the  subtests,  and  the  SDs  of  the  criterion  scores.  These  b-weights  applied 
to  the  operational  test  scores  would  provide  a  composite  that,  if  the  appropriate 
regression  constant  were  subtracted,  would  have  a  mean  of  50  and  a  SD  less  than  10 
(because  of  the  effects  of  the  positive  inter-correlation  coefficients  among  the 
subtests).  The  b-weights  are  computed,  ignoring  the  regression  constants,  as  follows: 

b- weight  =  B-weight  *  (SD)C  /  (SD)t , 

where  t  represents  a  subtest,  SDt  =10,  and  c  represents  the  criterion  variable. 

4.  The  composite  computed  in  step  3  will  have  a  SD  less  than  10.  We  wish  to  convert 
this  composite  to  have  a  SD  of  20.  To  do  this  we  will  multiply  each  b-weight  by  a 
composite  multiplier  (CM)  that  will  convert  the  composite  to  have  a  SD  of  20  without 
affecting  the  composite  mean.  CM  can  be  computed  as  follows. 

CM  =  20/  (10  *  (bRbT)1/2), 

where  b  is  a  vector  of  b-weights  and  R  is  the  Youth  Population  matrix  of  subtest 
inter-correlation  coefficients. 

5.  We  can  now  compute  the  u  and  k  values  (i.e.,  the  operational  subtest  weights)  for  each 
composite: 

Uj  =  CM  *  b-weight  of  the  j-th  subtest 
k  =  100  -  £  Uj  *  50 


Key  Formulae  From  Gulliksen 

The  algorithms  we  use  to  correct  for  restriction  in  range  due  to  “selection”  effects 
are  developed  and  described  by  Gulliksen  (1950)21.  His  development  is  based  on  a 
model  that  visualizes  the  presence  of  both  explicit  and  implicit  selection  processes  in  the 
unrestricted  population,  and  the  presence  of  both  explicitly  and  implicitly  selected 
variables  in  the  restricted  population.  Thus,  both  explicit  and  implicit  variables  are 
present  in  both  the  unrestricted  and  restricted  populations.  The  author  shows,  in  the 
context  of  this  model,  relationships  among  the  restricted  and  unrestricted  variances/co¬ 
variances  without  relaxing  flexibility  as  to  which  population  contains  the  unknowns  that 
cannot  be  directly  computed  but  can  be  determined  on  the  basis  of  the  relationships 
defined  in  his  model. 

The  Gulliksen  formulae  for  correcting  variances  and/or  co-variances  for 
restriction  in  range  effects  are  based  on  Lawley’s  (1943)  assumptions  that  include  the 
following:  (1)  that  the  regression  of  the  implicitly  restricted  variables  on  the  explicitly 
restricted  predictors  is  linear;  (2)  that  the  co-variance  of  the  restricted  variables  exhibit 
homoscedasticity;  and  (3)  that  the  G-weights  for  application  to  the  population  variance- 


21  See  H.  Gulliksen,  Theory  of  Mental  Tests.  New  York:  John  Wiley  &  Sons,  1950. 
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covariance  matrix  of  operational  test  scores  (explicitly  restricted  variables,  e.g.,  sub-tests) 
are  invariant  to  the  effects  of  restriction  (as  defined).  Thus  it  is  assumed  that 

G  =(Cxx)-‘  (Cxc)T 

can  be  computed  in  a  restricted  population  sample  and  substituted  in  formulae  for  use  in 
the  unrestricted  population  where  a  G- weight  is  to  be  entered.  Gulliksen’s  formula  42, 
used  to  compute  criterion  variance  in  the  Youth  Population,  requires  such  an  entry.  This 
criterion  variance  is  essential  for  converting  Beta-weights  into  b-weights  and  obviously 
cannot  be  directly  computed  in  the  Youth  Population. 

As  previously  stated,  our  objective  is  to  have  an  algorithm  replete  with  valid 
formulae  that  will  convert  operational  test  scores  into  LSEs  of  the  criterion  (i.e.  PP 
composites)  in  a  scale  appropriate  for  use  in  the  indicated  population. 

Application  of  Formulae  37  and  42 

Applying  combined  formulae  37  and  42  to  one  criterion  variable  at  a  time,  and 
making  small  changes  in  Gulliksen’s  notation,  we  can  compute  the  squared  SD  of  each 
Youth  Population  criterion  variable  associated  with  each  job  family.  This  result  can  be 
described  as  the  Youth  Population  criterion  variance,  or  YPCV : 

YPCY  =  1.0  +  Cxc  (Cxx)’1  (  (  *Cxx)  (Cxx)’1  - 1 )( Cx/, 

where  (  Cxq)  is  a  9  by  1  vector  of  co-variances  between  the  criterion  variable  and  each  of 
the  9  tests,  Cxx  is  a  9  by  9  matrix  of  co-variances  among  9  tests  using  the  operational  test 
scores,  and  vectors  are  denoted  by  underlining.  Note  that  the  asterisk  matrix,  e.g.,  *0, 
indicates  computation  in  the  unrestricted  (i.e.,  Youth  Population)  sample.22 

The  R  matrix  has  the  following  relationship  with  the  0^  matrix: 


R  =  SXCXX  Sx, 

where  S  is  a  diagonal  matrix  for  which  the  diagonal  elements  are  equal  to  the  reciprocals 
of  the  SDs  of  either  the  subtests  or  the  criterion  variable  in  either  the  MOS  sample  or  the 
Youth  Population,  as  indicated. 

The  *CxcT  matrix  is  derived  from  the  Gulliksen  formula  as: 

(*cxc)T  =  (*cxx)  (G)  =  (*cxx)  (cmy\cxc)T . 


22  Note  that  YPCV  can  also  be  written  as  follows: 

YPCV  =  1.0  +  (WT)(*C„ W  -  (CJT  ), 

where  W  =  (Cxx)’1  (CxJT  ,  a  9  by  1  vector  of  regression  weights  for  a  specified  job  family.  W  will  also  be 
recognized  as  one  column  of  the  G  matrix. 
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Note  that  one  column  of  *CxcT  is  (  CxJT ,  a  vector  used  in  the  computation  of  YPCV. 

The  validity  matrix  (*VT )  required  to  compute  Beta  weights  in  the  Youth  Population  has 
the  following  relationship  with  the  *CxCT  vector: 

one  column  of  *VT  is  (*SX )  (*CxC)T  (*SC )  , 
and  note  that  *SC  is  a  scalar. 

Positively  Weighted  Composites  for  the  Visible  Tier 

This  section  extends  the  initially  professed  objectives  of  this  appendix  beyond 
restriction  in  range  corrections  and  the  conversion  of  Betas  to  u  and  k  values.  We  will 
now  discuss  the  methodology  for  selecting  the  “best”  positively  weighted  composites 
where  best  is  defined  in  terms  of  maximizing  the  multiple  correlation  coefficient  of  a  set 
of  tests  with  the  criterion. 

The  surest  way  to  find  this  best  positively  weighted  composite  from  a  set  of  n 
tests  is  to  compute  the  Betas  and  validity  coefficients  for  every  possible  combination  of  n 
tests,  then  successive  levels:  for  n-1  tests,  then  n-2  tests, . .  .to  2  tests  —  rejecting  any 
combination  of  tests  that  has  one  or  more  negative  weights.  There  is  no  need  to  actually 
consider  all  of  these  combinations  since  there  comes  a  point  in  this  process  where  all 
multiple  correlation  coefficients  (Rs)  for  succeeding  levels  are  lower  than  the  highest  R 
in  a  prior  level. 

The  multiple-correlation  coefficient,  R,  corresponding  to  each  set  of  Betas  is 
computed  for  each  combination  whether  or  not  all  of  the  weights  are  positive.  Clearly,  if 
the  R  for  each  combination  of  m-1  tests,  negative  weights  permitted,  was  less  than  the 
highest  R  for  m  positively  weighted  subtests  computed  from  the  combinations  considered 
at  the  prior  level,  the  stopping  point  has  been  reached.  After  the  stopping  criterion  has 
been  reached,  the  set  of  subtests  with  all  positively  weighted  coefficients  that  provides 
the  maximum  R  is  selected  as  the  very  best  set  and  these  weights  become  the  B-weights 
for  the  associated  subtests.  All  other  tests  are  given  a  weight  of  zero  in  the  composite 
associated  with  the  specified  job  family. 
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